Data Integration: Inconsistency Detection and Resolution Based on Source Properties

This paper addresses the problem of integration of multiple heterogeneous information sources. The sources may conflict with each other on the following three levels: their schema, data representation, or data themselves. Most of the approaches in this area of research resolve inconsistencies among different schemas and data representations, and ignore the possibility of data-level conflict altogether. The few that do acknowledge its existence are mostly probabilistic approaches which just detect the conflict and provide a user with some additional information on the nature of the inconsistency (e.g. give a set of conflicting values with attached probabilities). We propose an extension to the relational data model that makes use of meta-data of the information sources called properties. This extension gives ground to a flexible data integration technique described in this work. The process of data integration for a particular user query consists of construction of the data blocks such that their extended union constitutes the query result, data conflict detection and data conflict resolution. An improvement to data clustering techniques in the conflict detection phase is also presented in the paper. It uses another type of meta-information available from the sources (source descriptions in terms of the virtual database schema) to narrow down the areas of possible data conflicts. For the conflict resolution phase, a flexible algorithm is offered. The algorithm is guided by user-defined importance (or weights) of properties and by expert-defined resolution strategies that incorporate the domain knowledge.